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Abstract Body 


Background: 

The adoption of the Common Core State Standards for Mathematics (CCSSM, 2010) in 
46 states has raised the mathematical proficiency bar for U.S. students. The advent of these new 
standards is at a time when the body of empirical evidence on the negative consequences of early 
and persistent difficulties in mathematics is clear, compelling, and growing. In an investigation 
of mathematics achievement data generated from 7,892 children in the Early Childhood 
Longitudinal Study-Kindergarten Cohort (ECLS-K), Morgan, Earkus, and Wu (2009), for 
example, found students with persistent math difficulties in both the fall and spring of 
kindergarten had a 70% chance of scoring in the same risk category at the end of fifth grade. 

The purpose of this 4-year Goal-3 Efficacy Trial, funded by the Institute of Education 
Sciences (Clarke, Doabler, Smolkowski, & Eien, 2012), is to study the efficacy of ROOTS, a 
(Tier 2) kindergarten mathematics intervention designed to improve the mathematics 
achievement of students at risk for MD. This presentation focuses on data generated from the 
first year of the larger efficacy trial (2012-2013). The study used a randomized control trial 
(RCT) design (blocking on classrooms) to investigate the ROOTS intervention when 
implemented under rigorous experimental conditions. Year 1 of the ROOTS efficacy trial took 
place in 29 kindergarten classrooms from four school districts in Oregon with approximately 10 
eligible students per classroom. Eligibility of student participation was based on screening scores 
from two standardized measures of mathematics achievement. The research team randomly 
assigned these 10 students to one of three conditions: (1) a ROOTS-large group (5:1 student- 
teacher ratio), (2) a ROOTS-small group (2:1 student-teacher ratio), and (3) a no-treatment 
control group. A total of 58 ROOTS groups participated in Year 1. 

Focus of Study: 

There are two primary aims of this presentation. Aim 1: Present impact findings from 
the first year of the 4-year ROOTS efficacy trial, Eor this first aim we will examine the 
immediate impact of the ROOTS curriculum on student mathematics achievement. During the 
2009-2010 school year, we conducted a preliminary investigation of the ROOTS curriculum in 
29 kindergarten classrooms and found promising effects on the mathematics achievement of 
students at risk for MD. We anticipate finding similar results in this efficacy trial. 

Aim 2: Empirically examine the association between instructional intensity, 
manipulated via instructional group size, and student mathematics achievement. As part of 
our theoretical rationale for examining instructional intensity, we use a treatment intensity 
framework specified by Warren, Yoder and Eey (2007), which centers on specifying the 
particulars - “active ingredients” - of the intervention that are hypothesized to improve student 
learning. To our knowledge, few, in any, studies on mathematics interventions have yet to 
directly examine the effect of manipulating instructional intensity within a single study. Here, we 
manipulate group size, a proxy of instructional intensity, to examine hypothesized mechanisms 
of change (i.e., student-teacher instructional interactions) in the ROOTS intervention. We 
considered ROOTS groups with a 5:1 student-teacher ratio as a “low intensity” treatment groups 
and ROOTS groups with a 2:1 student-teacher ratio as a “high intensity” treatment groups. 
Instructional intensity was measured using a low-inference observation instrument. 

Setting: 

Year 1 of this 4-year study was conducted in Oregon during the 2012-2013 school year. 

A total of 29 kindergarten classrooms from 14 schools were recruited to participated. These 
schools were from four school districts (urban and suburban), which had student enrollments that 
ranged from 5,000 to 38,000 students. Each of the 29 participating classrooms contained three 
instructional groups: (1) a low intensity ROOTS-large group (5:1 student-teacher ratio), (2) a 
high intensity ROOTS-small group (2:1 student-teacher ratio), and (3) a no-treatment control 
group. A total of 58 ROOTS groups participated in Year 1 (29 ROOTS-large, 29 ROOTS-small). 
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Participants: 

A total of 31 interventionists taught the 58 ROOTS groups. Interventionists, on average, 
taught more than one group (i.e., ROOTS-large and ROOTS-small) and eonsisted of distriet- 
employed instruetional assistants (n = 20) and temporary personnel (n = 11) hired by the Center 
on Teaehing and Learning at the University of Oregon for the purposes of the study. Two of the 
interventionists were kindergarten teaehers. Of the interventionists, 28 were female (90%) and 
the majority identified as Caueasian. 

Aeross the 29 kindergarten elassrooms, a total of 794 students were sereened for 
mathematies diffieulties and eligibility for the ROOTS intervention. Currently, all student and 
teaeher demographie data have been eolleeted, but the descriptive analyses have yet to be fully 
completed. Based on the screening scores, 10 students per classroom were selected to be in the 
study and thus eligible for randomization. A total of 291 students were randomly assigned to the 
three groups: ROOTS-large {n = 146), ROOTS-small {n = 58), and no-treatment control group {n 
= 87). The final analytic sample consisted of 252 students: ROOTS-large (n = 49), ROOTS- 
small (n = 124), and no-treatment control group (n = 79). 

ROOTS Intervention: 

ROOTS is a Tier 2 kindergarten intervention program that was designed to be delivered 
by interventionists in small-group instructional formats (i.e., 5:1 or 2:1 student-teacher ratios), 5 
times per week for approximately 10 weeks. The goal of ROOTS is to support students’ 
development of procedural fluency with and conceptual understanding of whole number 
concepts. The specific focus on whole number aligns with the heightened expectations of the 
CCSSM (2010) and calls from expert panels for more focused and coherent Tier 1 curricula 
(NMAP, 2008). ROOTS provides in-depth instruction in whole number concepts by linking the 
informal mathematical knowledge developed prior to school entry with the formal mathematical 
knowledge developed in kindergarten. The program includes 50 lessons, approximately 20 
minutes in duration. Each lesson consists of 4 to 5 brief math activities that center on three key 
areas of whole number understanding: (a) Counting and Cardinality (b) Number Operations and 
(c) Base 10/Place Value. Curricular objectives advance students from an initial understanding of 
whole number through more sophisticated aspects of whole numbers in kindergarten 
mathematics. 

A central feature of the ROOTS program is its explicit and systematic approach to 
instruction. This approach includes modeling and demonstrating what students will learn, 
providing guided practice opportunities, using visual representations of mathematics, and 
delivering academic feedback. For example, lessons provide interventionists with specific 
guidelines for demonstrating concepts and skills associated with whole numbers, and providing 
timely academic feedback to students as they engage in learning activities. The program also 
provides students with frequent and structured practice opportunities to promote procedural 
fluency and incorporates a host of visual representations to deepen conceptual understanding. 
Once students demonstrate initial proficiency with targeted math content, instructional scaffolds 
are systematically withdrawn to promote learner independence. Combined, the program’s design 
and delivery principles form an instructional base for interventionists to facilitate overt and 
dynamic instructional interactions among teachers and students around whole number concepts 
and skills. An important category of these instructional interactions is mathematical discourse or 
math verbalizations, which permit students to demonstrate their mathematical thinking and 
understanding (Gersten et al., 2009; NRC, 2001). 

Professional development. Participating interventionists attended two curriculum 
workshops for a total of 10 hours of professional development on the implementation of the 
ROOTS curriculum. The initial 6-hour workshop targeted the instructional objectives of Lessons 
1-25, the critical content of kindergarten mathematics (CCSSM, 2010), small-group management 
techniques, and the instructional practices that have been empirically validated to increase 
student math achievement (e.g., teacher provided academic feedback; Gersten et ah, 2009). A 
second 4-hour workshop was provided midway through the intervention and focused on the 
second half of the ROOTS curriculum. Lessons 26-50. 
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Research Design: 

The study design was a RCT, with students randomly assigned to eondition bloeking on 
elassrooms. We bloeked at the elassroom level to control for teacher effects on student outcomes. 
Our unit of analysis is instructional groups. In Year 1 a total of 58 ROOTS groups from 29 
classrooms in Oregon participated. Within each of&ese classrooms, 10 ROOTS eligible students 
were randomly assigned to one of three conditions : (1) a low intensity ROOTS-large group (5:1 
student-teacher ratio), (2) a high intensity ROOTS-small group (2:1 student-teacher ratio), and 
(3) a no-treatment control group {n = 3). Use of this type of research design allows us to address 
the efficacy of the ROOTS curriculum compared to a no-treatment control (Aim 1) and the 
efficacy of the ROOTS curriculum when the size of ROOTS small groups are experimentally 
manipulated to intensify the instructional interactions between teachers and student around 
critical whole number concepts (Aim 2). 

The Year 1 final analytic sample consisted of 252 students: ROOTS-large {n = 49), 
ROOTS-small {n = 124), and no-treatment control group {n = 79). Control students did not 
receive ROOTS instruction but were not prohibited from receiving standard district intervention 
services. All participants received district- approved (Tier 1) mathematics instruction. 

ROOTS Eligibility/Screening Process: 

To determine student eligibility for the ROOTS intervention all participating kindergarten 
students from the 29 classrooms were screened for MD using two standardized measures of early 
mathematics achievement: (1) Number Sense Brief Screener (Jordan et al. 2009) and the 
Assessing Student Proficiency in Early Number Sense (ASPENS) measures (Clarke, Gersten, 
Dimino, & Rolfhus, 2012). Based on the performances of the two screening measures, we 
calculated an overall standard score (z score) for each student. Within each classroom, students’ 
z-scores were ranked ordered from high to low. Compared to the full sample of kindergarten 
students assessed in the screening process {N= 794), the ROOTS eligible students performed in 
the low average to well-below average range (< 35th percentile). 

Data Collection and Analysis: 

Trained staff members collected all student and classroom observation data. Data 
collection met acceptable reliability criteria for all student and classroom measures included in 
the analysis. 

Student outcomes. Student mathematics performance measures included the Test of 
Early Mathematics Ability - 3rd Edition (TEMA-3), ASPENS, and the ROOTS Assessment of 
Early Numeracy Skills (RAENS), a measure considered proximal to the ROOTS intervention. 

All three measures were individually administered to ROOTS eligible students at pretest (Ti) and 
posttest (T 2 ) times. Each measure demonstrates strong psychometric properties. 

Direct observations. To measure instructional intensity, we used the Classroom 
Observations of Student-Teacher Interactions (COSTI) measure, a low-inference, frequency- 
based observation instrument developed by Smolkowski and Gunn (2012). Previous research on 
the COSTI documents preliminary evidence that its instructional behaviors are associated with 
gains in student reading (Smolkowski & Gunn, 2012) and mathematics (Doabler et ah, 2013) 
outcomes. The COSTI targets seven instructional behaviors: (a) teacher demonstrations, (b) 
guided practice opportunities for two or more students, (c) guided practice for individuals, (d) 
independent practice opportunities for two or more students, (e) independent practice 
opportunities for individuals, (f) student mistakes, and (g) teacher-provided academic feedback. 
Trained observers conducted all observations. Each ROOTS group was observed three times 
across the 10 intervention weeks 


It is important to note that with exception to group size, the treatment groups used the same 
intervention materials and adhered to the same implementation plan (i.e., one lesson per day, 5 
days per week for ~10 weeks). 
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Statistical analysis. To address Aim 1, we will analyze intervention effects on each of 
the outcome measures with a nested time by condition analysis (Murray, 1998) to test differences 
between conditions on change in outcomes from the start of the intervention (Ti) to the end of 
the intervention (T 2 ). This analysis approach will include all data — ^whether or not a student’s 
scores were present at both time points — to estimate differences between assessment times and 
between conditions. The nested time by condition analysis accounts autocorrelation among 
assessments within individual students and the intraclass correlation associated with multiple 
students nested within same schools. As a test of net differences, it also provides a more 
straightforward interpretation of the results. 

To address Aim 2, we will use a series of multilevel models, with students nested within 
classrooms, to test whether the instructional intensity data generated by the COSTI predicts 
student mathematics achievement (Raudenbush & Bryk, 2002). We will estimate these 
associations by regressing T 2 student-level math achievement scores (i.e., TEMA-3, ASPENS, 
and RAENS), adjusted for Ti scores, on each predictor, separately. Multilevel modeling will be 
conducted using HEM (Raudenbush, Bryk, Cheong, & Congdon, 2004), and parameters will be 
estimated using restricted maximum likelihood. To ease the interpretation of results, we will 
compute Pseudo-R 2 (Singer & Willett, 2003) as a measure of effect size for each fixed effect of 
the instructional intensity scores on math achievement. Pseudo-R 2 represents the decrease in 
classroom-level variance between unconditional and conditional models, or the proportion of 
classroom-level variance explained in the outcome measure by a predictor or set of predictors. 

Findings: 

Data analysis is currently underway. Eor the study’s first aim, we plan to investigate the 
impact of the ROOTS intervention on distal and proximal measures of student mathematics 
achievement. Preliminary analyses generated in an earlier investigation of the ROOTS 
intervention revealed small to medium effect sizes for the TEMA-3 (Pro-Ed, 2007) scaled score 
(Hedges’ g = 0.37, p = .037), EN-CBM total score (g = 0.30, p = .187). All effect sizes generated 
in this earlier study meet the What Works Clearinghouse Standards (WWC, 2013) for showing a 
“substantively important positive effect” (p. 60). Eor the current study, we expect to find similar 
results to those described above. Replication of these earlier findings would generate additional 
efficacy evidence for the ROOTS intervention. 

The second aim will investigate the association between student-teacher instructional 
interactions and student mathematics achievement, and test for differences between the ROOTS 
groups on instructional intensity scores (i.e., instructional intensity). Table 1 shows descriptive 
statistics for the first round of ROOTS observations with COSTI. While preliminary, these 
results suggest that students in the ROOTS-small group are receiving rates of practice 
opportunities that are greater than or equal to their at-risk peers in the ROOTS-large groups. 
Because the ROOTS-small groups have smaller student-teacher ratios one could argue that the 
practice opportunities taking place in these groups are more individualized and thus more 
intensive for students. 

Conclusion: 

There is convincing evidence that many students experience early and lasting difficulties 
in mathematics. This presentation will cover the Year 1 findings of a RCT involving the ROOTS 
Tier 2 intervention. ROOTS was implemented within 29 kindergarten classrooms from the state 
of Oregon. While preliminary, we will discuss possible implications of our findings in the 
context of multi-tiered delivery models of mathematics instruction. The findings from this study 
will be used to illustrate how research-based curricula can be used at Tier 2 to address the 
learning needs of at-risk students, and increase the intensity and frequency of student-teacher 
interactions around critical whole number content. As we delve into our data, we expect to (a) 
demonstrate initial efficacy of the ROOTS intervention and (b) unpack the nature and function of 
the student-teacher instructional interactions and determine their relevance for boosting the 
mathematics achievement of kindergarten students at risk for MD. 
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Appendix B, Tables and Figures 


Table 1. Descriptive Statistics for Round-1 of ROOTS Observations with COSTI 

ROOTS Group Size: Small (2:1) 


Variable - rate per minute 

M 

SD 

Min 

Max 

N 

Teacher model 

.41 

.22 

.06 

.83 

16 

Group guided practice 

.71 

.36 

.19 

1.43 

16 

Individual guided practice 

.33 

.32 

.00 

1.05 

16 

Group independent practice 

1.49 

.36 

.71 

2.05 

16 

Individual independent practice 

1.64 

1.18 

.52 

5.63 

16 

Mistake practice 

.14 

.14 

.00 

.43 

16 

Feedback practice 

1.51 

.78 

.48 

3.33 

16 

ROOTS Group Size: Large (5:1) 

Variable - rate per minute 

M 

SD 

Min 

Max 

N 

Teacher model 

A1 

.32 

.18 

1.56 

16 

Group guided practice 

.80 

.42 

.24 

1.67 

16 

Individual guided practice 

.24 

.22 

.00 

.68 

16 

Group independent practice 

1.26 

.66 

.37 

2.35 

16 

Individual independent practice 

1.25 

1.05 

.33 

4.67 

16 

Mistake practice 

.15 

.09 

.00 

.29 

16 

Feedback practice 

1.20 

.35 

.47 

1.75 

16 
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